1. Field of the Invention
The present invention relates to image processing methods, image processing apparatuses, and computer programs used therewith. More specifically, the present invention relates to an image processing method and apparatus that generate a high quality image by executing pixel-value correcting processing using an image captured with flash emission and an image captured without flash emission, estimating an optimal flash intensity, and performing correction based on the result of estimation, and to a computer program used therewith.
2. Description of the Related Art
Flashes, such as electrical flashes or strobes, are used as auxiliary light sources when cameras capture images. In recent years, digital still cameras (DSCs) have become rapidly spread. DSCs often use flash emission when capturing images. The use of flash emission enables various types of image capturing such as fill-in light (technique which, when a too dark shadow appears on a person's face, weakens the shadow), backlight correction (technique which, when an image of a person against the sun is captured, prevents the person's face from being darkened), catchlight (technique that captures a person's image having beautiful pupils by putting “twinkling light points” in the pupils of eyes), and daylight synchronization (technique that uses flashlight as auxiliary light in the daytime and in the evening). However, image capturing using flash emission may break color balance and may cause an overexposed highlight.
In general, in DSCs, white balance control is performed so that an image of a white subject is captured in white. For example, when image capturing is performed under a light-component environment in which light, such as natural light, illumination, or flashlight (strobe), emitted to a subject, has a high color temperature, and which has a strong blue component, sensitivity to the blue component is suppressed and sensitivity to a red component is relatively enhanced. Conversely, when image capturing is performed in a light-component environment in which light emitted to a subject has a low color temperature and which has a strong red component, white balance control is performed, such as suppressing the sensitivity to the red component and relatively enhancing the sensitivity to the blue component.
In white balance control, adjusting processing in which adjustment parameters are set in accordance with a light source for use in image capturing is normally executed. For example, when image capturing using flash emission is performed, white balance control is performed based on parameters in accordance with light components of flashlight in use.
Nevertheless, when, in a state in which there is external light other than flashlight, image capturing using flash emission is performed, two types of light, that is, the flashlight and the external light, are emitted onto a subject. Light reflected by the subject reaches an imaging element of a camera, thus causing image capturing. In the case of the above image capturing, by performing white balance control in accordance with flashlight, a part of the subject which receives a large amount of the flashlight is adjusted to have natural color. However, for an area whose image is captured as an image of reflected light caused by only the external light since the flash emission does not reach the imaging element of the camera, for example, a background image area, when white balance control is performed based on parameter settings in accordance with light components of the flashlight, no appropriate white balance control is executed, so that the area is output in a form having unnatural color.
Conversely, in the case of executing white balance control matching the background area, that is, white balance control on the entirety of the captured image on the assumption that only the external light is used for image capturing, a portion which receives a large amount of the flashlight is adjusted to have unnatural color.
Several configurations for coping the above problems have been proposed. For example, Japanese Unexamined Patent Application Publication No. 8-51632 discloses a configuration that performs acquiring an image captured without flash emission and an image captured with flash emission, dividing the two captured images into blocks, comparing luminances in each block, and, based on the result of comparison, implementing different white balance control in each block for the image captured with flash emission.
Regarding the white balance control, one of white balance control matching flashlight in each block, white balance control matching intermediate light between flashlight and external light, and white balance control matching external light is selected and executed. In this configuration, processing must be performed in units of blocks. Accordingly, when a problem causing block distortion occurs, and when a subject moves, it is impossible to perform appropriate processing.
Japanese Unexamined Patent Application Publication No. 2000-308068 discloses the following configuration. Specifically, at first, an image is captured with flash emission, the aperture opened, and the exposure time shortened. After that, an image is captured without using flash emission on an originally intended exposure condition. The former is called the “first image”, and the latter is called the “second image”. In the first image, pixels having values equal to or greater than a predetermined value are registered as a main subject area, and other pixels in the second image are registered as a background area. After that, white balance control on the first image is performed in accordance with the flash emission, and white balance control on the second image is performed in accordance with the external light. By combining the main subject area in the first image and the background area in the second image, a final recording image is formed.
However, in this configuration, it is impossible to perform white balance control on a subject that receives both the external light and the flashlight.
Japanese Unexamined Patent Application Publication No. 2000-307940 discloses a configuration including the configuration in Japanese Unexamined Patent Application Publication No. 2000-308068 and an image blurring detecting means. When the blurring is detected, the above first image is used as a recording image without being changed, and the first and second images are not combined. Accordingly, when the blurring is detected, in-image unnaturalness caused by a difference in color temperature between the flash emission and the external light cannot be eliminated.
Japanese Unexamined Patent Application Publication No. 8-340542 discloses a configuration that performs dividing the luminances of pixels of an image captured with flash emission by the luminances of corresponding pixels of an image captured without flash emission to find a contribution ratio of the flashlight, and, based on the contribution ratio, performing white balance control on the image captured with flash emission.
In this configuration, for an image captured with a mixture of flashlight and external light, white balance control parameters for the flashlight and the external light are simply interpolated based on the contribution ratio of flashlight, whereby a final image is generated. However, when considering a physical reflection model of light, a flashlight component and an external light component must be separately processed. Therefore, an optimal final image cannot be generated only by processing the image captured with the mixture of the flashlight and the external light.